1. Field of the Invention
The present invention relates to a system and method for supporting augmented reality in an ocean scene, and more particularly to a system and method for supporting augmented reality in an ocean scene, which can extract a camera motion in a state that it is difficult to perform camera auto-calibration using the existing method, by actively using prior knowledge about background geometry.
2. Background of the Related Art
Generally, in order to acquire geometrical information of an object from a moving image acquired through a camera and to acquire motion information (i.e., an external variable) and status information (i.e., an internal variable) of the camera during image capturing, a camera calibration process for estimating variables between image information acquired from the camera and real geometrical information of the object is essentially required. In the case of acquiring values of the external variable (i.e., moving information of the camera) and the internal variable (i.e., status information of the camera) of the camera during image capturing, the easiest and most accurate method that can be used is to obtain information of the camera by hardware using a motion control camera (MCC). However, this method has problems in that it is impossible to apply the method to the previously captured images and the MCC equipment for implementing the method is very expensive, and has drawbacks in that it is difficult to use the method on the filming spot such as sea background. Accordingly, methods for performing camera auto-calibration by software on the basis of image processing techniques have been researched. These methods are classified into ‘a method using an artificial calibration object’ and ‘a self-calibration method using natural features’.
Here, the method using an artificial calibration object is a method of performing auto calibration by capturing an image of a calibration tool having the shape of a three-dimensional (3D) rectangular parallelepiped and solving the geometrical relation between the shape in space of the rectangular parallelepiped and the projective image. Although this method has been widely used up to now and has the advantage that it produces a relatively accurate result, it has limitations in that the calibration object has the characteristic of a typical rectangular parallelepiped and the shape of the calibration object should be caught in an image sequence of which the auto calibration is to be performed. As a result, this method is not suitable as the camera auto-calibration method for the synthesis of real scenes, and moreover, it is impossible to perform the method in a sea background to be handled according to the present invention.
On the other hand, the self-calibration method is a method of performing the calibration by extracting natural feature points from images captured at various angles and performing the calibration using the corresponding relations among the natural feature points. According to this method, if the positions of the corresponding points are once confirmed in the respective frames, all frames are mated by twos or threes, and projective reconstruction is performed using a fundamental or essential matrix or a trifocal tensor. Thereafter, on the basis of the assumption for an “image of absolute conic (IAC)” method proposed by Richard Hartley, a camera internal variable matrix is obtained, and then a transform matrix for transforming a camera matrix into a metric space from a projective space is obtained. Since this method does not use the artificial calibration object as in the ‘method using an artificial calibration object’, it is useful for the real scene synthesis. However, noise is added in the process of finding relations among the corresponding points in the respective frames, and this may cause the performance to deteriorate. In practice, an LM (Levenberg Marquardt) algorithm should be inserted into many parts in order to make a robust system. However, in the sea image to be handled according to the present invention, the texture on the sea surface is successively changed due to waves, and thus it is difficult to extract the natural feature points consistently observed in many frames enough to make the auto calibration possible. In order to ensure the performance of the auto calibration as described above, at least one hundred consistent natural feature points are required. However, since it is impossible to extract such many consistent natural feature points in the sea image, it is also impossible to apply the self-calibration method to the sea image.
On the recent filming spots, it is general that an assistance records camera status information such as a focal length for the following computer graphics (CG) work. However, since it is expected that an HD camera to be produced in the future is provided with a function of simultaneously recording the focal length data, it would be possible to implement an effective CG/real scene synthesis system, as avoiding the process of predicting even the camera internal variables such as the auto-calibration as described above.